Deep learning-enabled 3D multimodal fusion of cone-beam CT and intraoral mesh scans for clinically applicable tooth-bone reconstruction

High-fidelity three-dimensional (3D) models of tooth-bone structures are valuable for virtual dental treatment planning; however, they require integrating data from cone-beam computed tomography (CBCT) and intraoral scans (IOS) using methods that either error-prone or time-consuming. Hence, this study presents Deep Dental Multimodal Fusion (DDMF), an automatic multimodal framework reconstructs 3D CBCT IOS. Specifically, the DDMF comprises IOS segmentation modules as well a reconstruction module with novel pixel representation learning architectures, prior knowledge-guided losses, geometry-based fusion techniques. Experiments on real-world large-scale datasets revealed achieved superior performance IOS, achieving 0.17 mm average symmetric surface distance (ASSD) substantial processing time reduction. Additionally, clinical applicability studies have demonstrated DDMF’s potential accurately simulating throughout orthodontic process.